Information processing apparatus and control method of information processing apparatus

ABSTRACT

An information processing apparatus includes: an operation unit which receives an operation; a first detection unit which detects a direction of the operation unit; an input and output conversion unit which converts an input of the operation received by the operation unit into an output based on a first regulation, when the direction of the operation unit is in a first state, and converts the input into an output based on a second regulation which is different from the first regulation, when the direction of the operation unit is in a second state; a second detection unit which detects an apparatus state which is at least one of a position and a direction of the information processing apparatus; and a control processing unit which performs a control process of the information processing apparatus, based on the output from the input and output conversion unit and the apparatus state.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Continuation of application Ser. No. 14/555,943 filed Nov. 28,2014, which claims priority to Japanese Patent Application JP2013-257675 filed Dec. 13, 2013. The disclosures of the priorapplications are hereby incorporated by reference herein in theirentirety.

BACKGROUND

1. Technical Field

The present invention relates to an information processing apparatus.

2. Related Art

A head mounted display (HMD) which is a display device to be mounted onthe head has been known as one information processing apparatus. Thehead mounted display, for example, generates image light showing animage using a liquid crystal display and a light source, and introducesthe generated image light to the eyes of a user using a projectionoptical system or a light guide plate, to allow a user to visuallyrecognize a virtual image. An operation with buttons or a track pad,movement of the head of a user detected by various sensors, and the likeare known as means for controlling the head mounted display.

JP-A-2011-82781 discloses a head mounted display in which a gyro sensoris embedded in a remote controller which is an operation unit, and thehead mounted display is operated according to an angular velocitydetected by the gyro sensor. In addition, JP-A-5-305181 discloses a gamemachine which allows a plurality of players to play the same game, andallows easy sterilization of a head mounted display by detachablyattaching the head mounted display to a main body of the game machine.

However, in the head mounted display disclosed in JP-A-2011-82781, theoperation of the head mounted display may be performed with the gyrosensor embedded in the operation unit, but it is difficult to mount asensor other than the gyro sensor embedded in the operation unit inorder to perform an operation different from the operation performedwith the angular velocity detected by the gyro sensor, according to adetected result of the other sensor. In addition, depending on anoperation system (hereinafter, also simply referred to as an “OS”), itis difficult to perform a plurality of control operations correspondingto each detected result, with respect to the detected results from theplurality of sensors, and it is difficult to perform the plurality ofcontrol operations without changing the OS. The problems described aboveare not limited to the head mounted display, but commonly occur in aninformation processing apparatus including an operation unit.

SUMMARY

An advantage of some aspects of the invention is to solve at least apart of the problems described above, and the invention can beimplemented as the following forms.

(1) An aspect of the invention provides an information processingapparatus. The information processing apparatus includes: an operationunit which receives an operation; a first detection unit which detects adirection of the operation unit; an input and output conversion unitwhich converts an input of the operation received by the operation unitinto an output based on a first regulation, when the direction of theoperation unit is in a first state, and converts the input into anoutput based on a second regulation which is different from the firstregulation, when the direction of the operation unit is in a secondstate which is different from the first state; a second detection unitwhich detects an apparatus state which is at least one of a position anda direction of the information processing apparatus; and a controlprocessing unit which performs a control process of the informationprocessing apparatus, based on the output from the input and outputconversion unit and the apparatus state. With this configuration, sincethe different control operations are performed in accordance withdetected results of a plurality of detection units, it is possible toperform various control operations with respect to the informationprocessing apparatus.

(2) In the information processing apparatus of the aspect describedabove, the first detection unit may be disposed in the operation unitand may detect the direction of the operation unit using an accelerationof the operation unit. With this configuration, since the controloperation of the information processing apparatus is performed based onthe input using the direction of the detected acceleration as areference, and not the direction of the operation unit, a user canperform the input using the direction of the detected acceleration as areference, and user operability is improved.

(3) In the information processing apparatus of the aspect describedabove, the first state and the second state may be determined based on arelationship between the direction of the operation unit and thedirection of gravity. With this configuration, since the controloperation of the information processing apparatus is performed based onthe direction of the operation unit and the detected direction of agravitational acceleration, a user can perform the input using thedetected direction of the gravitational acceleration as a reference, anduser operability is improved.

(4) In the information processing apparatus of the aspect describedabove, the first state may be a state where an operation angle formed bythe direction of the operation unit and the direction of gravity isequal to or smaller than 45 degrees, and the second state may be a statewhere the operation angle is greater than 45 degrees. With thisconfiguration, since the range of the angle to be converted based on thefirst regulation is great, the number of times of corrections forconverting the input received by the operation unit into the output islimited, it is easy to perform the control operation for converting theinput into the output, it is possible to decrease the load of theinformation processing apparatus, and it is possible to reduce afrequency of detecting the direction of the operation unit.

(5) In the information processing apparatus of the aspect describedabove, the second state may include a third state where the operationangle is greater than 45 degrees and equal to or smaller than 135degrees, and a fourth state where the operation angle is greater than135 degrees, and the input and output conversion unit may convert theinput of the operation received by the operation unit into an outputbased on a third regulation different from the first regulation, in thecase of the third state, and may convert the input into an output basedon a fourth regulation different from the first regulation and the thirdregulation, in the case of the fourth state. With this configuration,since the converted angle is divided into a limited number, it is easyto correct the input received by the operation unit to the output, it ispossible to decrease the load of the information processing apparatus,and it is possible to reduce the frequency of detecting the direction ofthe operation unit.

(6) The information processing apparatus of the aspect described abovemay further include an image display unit which forms image light basedon image data and allows a user to visually recognize the image light asa virtual image, in a state of being mounted on the head of a user, theapparatus state may be at least one of a position and a direction of theoperation unit or the image display unit, and the control processingunit may control the image light formed by the image display unit, asthe control process. With this configuration, an image or externalscenery visually recognized by a user differs depending on the state ofa user, and user convenience is improved.

(7) In the information processing apparatus of the aspect describedabove, the second detection unit may be disposed in the image displayunit and may detect at least one of the position and the direction ofthe image display unit, and the control processing unit may set aposition of the image light formed in the image display unit, as thecontrol process. With this configuration, since an image or the externalscenery visually recognized by a user differs depending on the change ofa visual line direction or the direction of the head of a user, userconvenience is further improved.

(8) In the information processing apparatus of the aspect describedabove, the first detection unit may detect the direction of theoperation unit with a frequency smaller than once every 0.5 seconds.With this configuration, it is not necessary to constantly detect theacceleration acting on the operation unit, and it is possible tosuppress the load applied to the information processing apparatus toimprove user convenience.

(9) In the information processing apparatus of the aspect describedabove, the control processing unit may include an operating system, andthe input and output conversion unit may a device driver or middleware.With this configuration, since the first detection unit and the inputand output conversion unit corresponding to the operation unit are used,the load relating to the control operation of the information processingapparatus is decreased, it is not necessary to change the software suchas an operating system, and it is possible to reduce a developmentperiod of the information processing apparatus.

All of the plurality of constituent elements included in each aspect ofthe invention are not compulsory, and modification, removal, replacementwith other new constituent elements, and removal of partial limitedcontent can be suitably performed regarding the constituent elementswhich are a part of the plurality of constituent elements, in order tosolve some or all of the problems described above or in order to realizesome or all of the advantages described in the specification. Some orall of the technical features included in one aspect of the inventiondescribed above can be combined with some or all of the technicalfeatures included in the other aspect of the invention described aboveto serve as one independent aspect of the invention, in order to solvesome or all of the problems described above or in order to realize someor all of the advantages described in the specification.

For example, one aspect of the invention can be realized as an apparatusincluding one or more or all of the five elements which are theoperation unit, the first detection unit, the input and outputconversion unit, the second detection unit, and the control processingunit. That is, this apparatus may or may not include the operation unit.The apparatus may or may not include the first detection unit. Theapparatus may or may not include the input and output conversion unit.The apparatus may or may not include the second detection unit. Theapparatus may or may not include the control processing unit. Theoperation unit, for example, may receive the operation. The firstdetection unit, for example, may detect the direction of the operationunit. The input and output conversion unit, for example, may convert aninput of the operation received by the operation unit into an outputbased on a first regulation, when the direction of the operation unit isin a first state, may convert the input into an output based on a secondregulation which is different from the first regulation, when thedirection of the operation unit is in a second state which is differentfrom the first state. The second detection unit, for example, may detectan apparatus state which is at least one of a position and a directionof the information processing apparatus. The control processing unit,for example, may perform a control process of the information processingapparatus, based on the output from the input and output conversion unitand the apparatus state. This apparatus, for example, can be realized asthe information processing apparatus, but can also be realized as anapparatus other than the information processing apparatus. According tothe aspect, it is possible to solve at least one problem among variousproblems, such as improvement and ease of operability of the device,integration of the device, or improvement of user convenience of theapparatus. Some or all of the technical features of each aspect of theinformation processing apparatus can be applied to this apparatus.

The invention can be realized as various aspects other than theinformation processing apparatus. For example, the invention can berealized in the aspect of a head mounted display, a control method of aninformation processing apparatus or a head mounted display, aninformation processing system, a head mounted type display system, acomputer program for realizing an information processing system or ahead mounted type display system, a recording medium with the computerprogram recorded therein, and a data signal including the computerprogram realized in a carrier wave.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is an explanatory diagram showing an appearance configuration ofa head mounted display.

FIG. 2 is a functional block diagram showing a configuration of the headmounted display.

FIG. 3 is an explanatory diagram showing a state of image light emittedby an image light generation unit.

FIG. 4 is an explanatory diagram showing a flow of a display imagecontrol process.

FIGS. 5A and 5B are explanatory diagrams showing schematic visual fieldswhich are visually recognized by a user before and after changing avisual line direction.

FIG. 6 is an explanatory diagram showing a flow of an operation inputprocess.

FIG. 7 is an explanatory diagram of a change of a display position of apointer by a pointer operation.

FIG. 8 is an explanatory diagram of a change of a display position ofthe pointer by a pointer operation.

FIGS. 9A and 9B are explanatory diagrams showing appearanceconfigurations of a head mounted display of Modification Examples.

DESCRIPTION OF EXEMPLARY EMBODIMENTS A. Embodiment A-1. Configuration ofHead Mounted Display

FIG. 1 is an explanatory diagram showing an appearance configuration ofa head mounted display 100. The head mounted display 100 is a displaydevice to be mounted on the head, and is also called a head mounteddisplay (HMD). The head mounted display 100 of the embodiment is anoptical transmission type head mounted display which allows a user tovisually recognize a virtual image and to directly visually recognizeexternal scenery at the same time. In the present specification, thevirtual image which is visually recognized by a user using the headmounted display 100 is also referred to as a “display image” for thesake of convenience. In addition, emission of image light generatedbased on image data is also referred to as “display of an image”.

The head mounted display 100 includes an image display unit 20 whichallows a user to visually recognize a virtual image in a state of beingmounted on the head of a user, and a control unit 10 (controller 10)which controls the image display unit 20.

The image display unit 20 is a body to be mounted on the head of a user,and has the shape of glasses in the embodiment. The image display unit20 includes a right holding unit 21, a right display driving unit 22, aleft holding unit 23, a left display driving unit 24, a right opticalimage display unit 26, a left optical image display unit 28, and a10-axis sensor 66. The right optical image display unit 26 and the leftoptical image display unit 28 are disposed so as to be positioned infront of right and left eyes of a user when the image display unit 20 ismounted thereon. One end of the right optical image display unit 26 andone end of the left optical image display unit 28 are connected to eachother in a position between the eyebrows of a user when the imagedisplay unit 20 is mounted thereon.

The right holding unit 21 is a member which is provided extending froman end portion ER which is the other end of the right optical imagedisplay unit 26 to a position corresponding to a temple of a user whenthe image display unit 20 is mounted on a user. In the same manner asdescribed above, the left holding unit 23 is a member which is providedextending from an end portion EL which is the other end of the leftoptical image display unit 28 to a position corresponding to a temple ofa user when the image display unit 20 is mounted thereon. The rightholding unit 21 and the left holding unit 23 hold the image display unit20 on the head of a user, as temples do for glasses.

The right display driving unit 22 and the left display driving unit 24are disposed at a side facing the head of a user when the image displayunit 20 is mounted thereon. Hereinafter, the right holding unit 21 andthe left holding unit 23 are also collectively simply referred to as the“holding units”, the right display driving unit 22 and the left displaydriving unit 24 are also collectively simply referred to as the “displaydriving units”, and the right optical image display unit 26 and the leftoptical image display unit 28 are also collectively simply referred toas the “optical image display units”.

The display driving units 22 and 24 include liquid crystal displays 241and 242 (hereinafter, also referred to as “LCDs 241 and 242”),projection optical systems 251 and 252, and the like (see FIG. 2). Theconfiguration of the display driving units 22 and 24 will be describedin detail later. The optical image display units 26 and 28 as opticalmembers include light guide plates 261 and 262 (see FIG. 2) and dimmingplates. The light guide plates 261 and 262 are formed of a lighttransmissive resin material and the like, and guide image light outputfrom the display driving units 22 and 24 to eyes of a user. The dimmingplates are optical elements having a thin plate shape, and are disposedso as to cover the front side of the image display unit 20 which is aside opposite the side of eyes of a user. The dimming plates protect thelight guide plates 261 and 262, and suppress damage or attachment ofdirt on the light guide plates 261 and 262. In addition, by adjustingthe light transmittance of the dimming plates, an amount of externallight entering eyes of a user can be adjusted, and a virtual image canbe adjusted so as to be easily visually recognized. The dimming platescan be omitted.

The 10-axis sensor 66 is a sensor which detects an acceleration (threeaxes), an angular velocity (three axes), geomagnetism (three axes), andan atmosphere (one axes). The 10-axis sensor 66 is embedded in the imagedisplay unit 20 in a position near the display driving unit 22, anddetects a movement or a position of the head of a user (hereinafter,also simply referred to as a “state of the image display unit 20”), whenthe image display unit 20 is mounted on the head of a user. The state ofthe image display unit 20 detected by the 10-axis sensor 66 correspondsto an apparatus state in the claims. The apparatus state of theinformation processing apparatus in the claims includes at least one ofa position and a direction of a part of the information processingapparatus. For example, a state where the position of the image displayunit 20 which is a part of the head mounted display 100 is changed andthe position and the direction of the device other than the imagedisplay unit 20 (for example, control unit 10) of the head mounteddisplay 100 are not changed, corresponds to a case in the claims wherethe position of the information processing apparatus is changed.

The image display unit 20 further includes a connection unit 40 forconnecting the image display unit 20 to the control unit 10. Theconnection unit 40 includes a main body cord 48 connected to the controlunit 10, a right cord 42, a left cord 44, and a coupling member 46. Theright cord 42 and the left cord 44 are two cords branched from the mainbody cord 48. The right cord 42 is inserted into a housing of the rightholding unit 21 from an end portion AP of the right holding unit 21 inan extension direction, and is connected to the right display drivingunit 22. In the same manner as described above, the left cord 44 isinserted into a housing of the left holding unit 23 from an end portionAP of the left holding unit 23 in an extension direction, and isconnected to the left display driving unit 24. The coupling member 46 isprovided at a branched point of the main body cord 48, and the rightcord 42 and the left cord 44, and includes a jack for connecting anearphone plug 30. A right earphone 32 and a left earphone 34 extend fromthe earphone plug 30.

The image display unit 20 and the control unit 10 transmit varioussignals through the connection unit 40. An end portion of the main bodycord 48 at a side opposite the coupling member 46, and the control unit10 include connectors (not shown) fitted to each other, respectively.The control unit 10 and the image display unit 20 are connected to eachother or disconnected from each other, by fitting of the connector ofthe main body cord 48 and the connector of the control unit 10 to eachother or releasing fitting thereof from each other. Metal cables oroptical fiber can be used as the right cord 42, the left cord 44, andthe main body cord 48, for example.

The control unit 10 is a device for controlling the head mounted display100. The control unit 10 includes a determination key 11, a lightingunit 12, a display switching key 13, a track pad 14, a luminanceswitching key 15, a direction key 16, a menu key 17, a power switch 18,and an acceleration sensor 19. The determination key 11 detects apressing operation and outputs a signal for determining content operatedin the control unit 10. The lighting unit 12 notifies an operation stateof the head mounted display 100 with a light emitting state thereof. Theoperation state of the head mounted display 100 is, for example, the ONor OFF state of the power. A light emitting diode (LED) is used as thelighting unit 12, for example. The display switching key 13 detects apressing operation and outputs a signal for switching the display modeof a moving content image to 3D or 2D, for example. The track pad 14detects an operation of a finger of a user on an operation surface ofthe track pad 14 and outputs a signal corresponding to the detectedcontent. Various track pads such as electrostatic, pressure-detectiontype, or optical track pads can be used as the track pad 14. Theluminance switching key 15 detects a pressing operation and outputs asignal for increasing or decreasing luminance of the image display unit20. The direction key 16 detects a pressing operation for a keycorresponding to the up, down, right, and left directions and outputs asignal corresponding to the detected content. The power switch 18detects a sliding operation of a switch to switch a power state of thehead mounted display 100. The acceleration sensor 19 acquires theacceleration (for example, gravitational acceleration) acting on thecontrol unit 10. In the embodiment, the acceleration sensor 19 acquiresthe acceleration acting on the control unit 10, on a regular basis, onceevery 0.5 seconds.

FIG. 2 is a functional block diagram showing a configuration of the headmounted display 100. As shown in FIG. 2, the control unit 10 includes astorage unit 120, a power 130, an operation unit 135, a CPU 140, aninterface 180, a transmission unit 51 (Tx 51), and a transmission unit52 (Tx 52). The operation unit 135 receives an operation from a user,and is configured with the determination key 11, the display switchingkey 13, the track pad 14, the luminance switching key 15, the directionkey 16, the menu key 17, the power switch 18, and the accelerationsensor 19.

The power 130 supplies power to each unit of the head mounted display100. A secondary battery can be used as the power 130, for example. Thestorage unit 120 stores various computer programs. The storage unit 120is configured with a ROM or a RAM. The CPU 140 reads out and executesthe computer programs stored in the storage unit 120 and functions as anoperating system 150 (OS 150), a display control unit 190, a directiondetermination unit 166, an image processing unit 160, a sound processingunit 170, and an input and output conversion unit 169.

The OS 150 used in the embodiment is Android (trademark). Using Android,it is difficult to perform a plurality of control operations accordingto results detected from a plurality of sensors. In the embodiment,Android is used as the OS 150, but another OS may be used in the otherembodiment.

The display control unit 190 generates a control signal for controllingthe right display driving unit 22 and the left display driving unit 24.In detail, the display control unit 190 separately controls ON and OFFstates of driving of the right LCD 241 by a right LCD control unit 211,the ON and OFF states of the driving of a right backlight 221 by a rightbacklight control unit 201, the ON and OFF states of the driving of theleft LCD 242 by a left LCD control unit 212, and the ON and OFF statesof the driving of a left backlight 222 by a left backlight control unit202, with the control signals. Accordingly, the display control unit 190controls generation and emission of the image light by each of the rightdisplay driving unit 22 and the left display driving unit 24. Forexample, the display control unit 190 generates the image light for bothof the right display driving unit 22 and the left display driving unit24, generates the image light for one thereof, or does not generate theimage light for both thereof. The display control unit 190 transmits therespective control signals with respect to the right LCD control unit211 and the left LCD control unit 212 through the transmission units 51and 52. In addition, the display control unit 190 transmits therespective control signals with respect to the right backlight controlunit 201 and the left backlight control unit 202.

The direction determination unit 166 specifies a visual line directionof a user, based on the direction of the image display unit 20 specifiedby the 10-axis sensor 66. In the embodiment, in a case where the visualline direction is changed, the direction determination unit 166determines whether or not the change in an angle formed by the initialvisual line direction and the changed visual line direction in apredetermined unit period (hereinafter, also referred to as a “changedangle”) is equal to or greater than a threshold value. The directiondetermination unit 166 and the 10-axis sensor 66 correspond to a seconddetection unit in the claims.

The image processing unit 160 acquires an image signal included in thecontent. The image processing unit 160 separates a synchronizationsignal such as a vertical synchronization signal VSync or a horizontalsynchronization signal HSync from the acquired image signal. Inaddition, the image processing unit 160 generates a clock signal PCLK inaccordance with the separated vertical synchronization signal VSync orthe horizontal synchronization signal HSync, by using a phase lockedloop (PLL) circuit (not shown) or the like. The image processing unit160 converts an analog image signal with the separated synchronizationsignal into a digital image signal using an A/D conversion circuit (notshown) or the like. After that, the image processing unit 160 stores theconverted digital image signal in a DRAM in the storage unit 120 foreach frame, as image data (RGB data) of a target image. The imageprocessing unit 160, if necessary, may execute an image process such asa resolution conversion process, various color tone correction processsuch as adjustment of luminance and chroma, or a keystone correctionprocess, with respect to the image data.

The image processing unit 160 transmits the generated clock signal PCLK,the vertical synchronization signal VSync, the horizontalsynchronization signal HSync, and the image data Data stored in the DRAMin the storage unit 120 through the transmission units 51 and 52. Theimage data Data transmitted through the transmission unit 51 is alsoreferred to as “image data for the right eye”, and the image data Datatransmitted through the transmission unit 52 is also referred to as“image data for the left eye”. The transmission units 51 and 52 functionas a transceiver for serial transmission between the control unit 10 andthe image display unit 20.

In addition, the image processing unit 160 performs various controloperations of a display image to be displayed on the image display unit20, based on a visual line direction of a user specified by thedirection determination unit 166. In the embodiment, the imageprocessing unit 160 controls whether or to not display the imagedisplayed on the image display unit 20, according to the changed angleof the visual line direction determined by the direction determinationunit 166. For example, when the changed angle of the visual linedirection is equal to or greater than the threshold value, the imageprocessing unit 160 does not display the display image displayed on theimage display unit 20.

The sound processing unit 170 acquires a sound signal included in thecontent, amplifies the acquired sound signal, and supplies the soundsignal to a speaker (not shown) in the right earphone 32 and a speaker(not shown) in the left earphone 34 connected to the coupling member 46.For example, in the case that a Dolby (trademark) system is used, aprocess with respect to the sound signal is performed, and differentsounds with changed frequencies, for example, are output from the rightearphone 32 and the left earphone 34.

The input and output conversion unit 169 is a device driver whichacquires a signal according to the operation input from a user and asignal indicating an acceleration acting on the control unit 10 in whichthe operation unit 135 is formed, and transmits an output signal to theOS 150 based on the acquired signals. As the signal according to theoperation input, an operation input with respect to the track pad 14,the direction key 16, and the power switch 18 of the operation unit 135is used, for example. The input and output conversion unit 169 specifiesa direction of the operation unit 135 and the direction of gravity asthe inputs, based on the acceleration acquired by the accelerationsensor 19. The input and output conversion unit 169 determines aconversion angle for converting the inputs into the output signal, basedon an angle formed by the direction of the operation unit 135 and thedirection of gravity. The conversion angle is an angle for correctingthe angle formed by the direction of the operation unit 135 and thedirection of gravity as the inputs, when the input and output conversionunit 169 transmits the output signal to the OS 150. The input and outputconversion unit 169 converts the inputs received by the track pad 14 ofthe operation unit 135 into the output signal based on the conversionangle, and transmits the converted output signal to the OS 150. Thesignal acquired by the acceleration sensor 19 is transmitted to theinput and output conversion unit 169 and is not transmitted to the OS150. The conversion angle will be described in detail. The conversionangle corresponds to a first regulation and a second regulation in theclaims. The acceleration sensor 19 and the input and output conversionunit 169 correspond to a first detection unit in the claims.

The interface 180 (FIG. 2) is an interface for connecting variousexternal devices OA which are supply sources of the content, to thecontrol unit 10. As the external devices OA, a personal computer (PC), amobile phone, a game terminal, and the like are used, for example. Asthe interface 180, a USB interface, a micro USB interface, an interfacefor a memory card, or the like can be used, for example.

The image display unit 20 includes the right display driving unit 22,the left display driving unit 24, the right light guide plate 261 as theright optical image display unit 26, the left light guide plate 262 asthe left optical image display unit 28, and the 10-axis sensor 66.

The right display driving unit 22 includes a reception unit 53 (Rx 53),the right backlight control unit 201 (right BL control unit 201) and theright backlight 221 (right BL 221) functioning as light sources, theright LCD control unit 211 and the right LCD 241 functioning as displayelements, and the right projection optical system 251. The rightbacklight control unit 201 and the right backlight 221 function as thelight sources. The right LCD control unit 211 and the right LCD 241function as the display elements. The right backlight control unit 201,the right LCD control unit 211, the right backlight 221, and the rightLCD 241 are also collectively referred to as an “image light generationunit”.

The reception unit 53 functions as a receiver for serial transmissionbetween the control unit 10 and the image display unit 20. The rightbacklight control unit 201 drives the right backlight 221 based on theinput control signal. The right backlight 221 is a luminescent body suchas an LED or an electroluminescence (EL) device, for example. The rightLCD control unit 211 drives the right LCD 241 based on the clock signalPCLK, the vertical synchronization signal VSync, the horizontalsynchronization signal HSync, and the image data for a right eye inputthrough the reception unit 53. The right LCD 241 is a transmission typeliquid crystal panel in which a plurality of pixels are disposed in amatrix form.

The right projection optical system 251 is configured with a collimatinglens which converts the image light emitted from the right LCD 241 intoa luminous flux in a parallel state. The right light guide plate 261 asthe right optical image display unit 26 reflects the image light outputfrom the right projection optical system 251 along a predetermined lightpath and guides the reflected image light to the right eye RE of a user.The right projection optical system 251 and the right light guide plate261 are also collectively referred to as a “light guide unit”.

The left display driving unit 24 has the same configuration as that ofthe right display driving unit 22. The left display driving unit 24includes a reception unit 54 (Rx 54), the left backlight control unit202 (left BL control unit 202) and the left backlight 222 (left BL 222)functioning as light sources, the left LCD control unit 212 and the leftLCD 242 functioning as display elements, and the left projection opticalsystem 252. The left backlight control unit 202 and the left backlight222 function as the light sources. The left LCD control unit 212 and theleft LCD 242 function as the display elements. The left backlightcontrol unit 202, the left LCD control unit 212, the left backlight 222,and the left LCD 242 are collectively referred to as an “image lightgeneration unit”. The left projection optical system 252 is configuredwith a collimating lens which converts the image light emitted from theleft LCD 242 into a luminous flux in a parallel state. The left lightguide plate 262 as the left optical image display unit 28 reflects theimage light output from the left projection optical system 252 along apredetermined light path and guides the reflected image light to theleft eye LE of a user. The left projection optical system 252 and theleft light guide plate 262 are also collectively referred to as a “lightguide unit”.

FIG. 3 is an explanatory diagram showing a state of image light emittedby the image light generation unit. The right LCD 241 drives the liquidcrystal in each pixel position disposed in a matrix form to change thetransmittance of the light passing through the right LCD 241, andaccordingly modulates illumination light IL emitted from the rightbacklight 221 into effective image light PL showing an image. The leftimage light generation unit has the same configuration. As shown in FIG.3, the backlight system is used in the embodiment, but the image lightmay be emitted using a front light system or a reflection system.

A-2. Display Image Control Process

FIG. 4 is an explanatory diagram showing the flow of a display imagecontrol process. The display image control process is a process in whicha display image displayed on the image display unit 20 is controlledaccording to the changed angle of the visual line direction of a userspecified by the 10-axis sensor 66 and the direction determination unit166 and the output signal converted by the input and output conversionunit 169, after the image is displayed on the image display unit 20.

In the display image control process, first, the image display unit 20displays an image based on the image signal transmitted from the imageprocessing unit 160 (Step S11). Next, the 10-axis sensor 66 detects astate of the image display unit 20 (Step S12). In the embodiment, the10-axis sensor 66 detects the acceleration of the image display unit 20and then the control of the display image according to the detectedacceleration is performed. Next, the direction determination unit 166specifies a visual line direction of a user based on the detectedacceleration (Step S13). Then, the direction determination unit 166determines whether or not the visual line direction is changed (StepS14). When the changed angle of the visual line direction is equal to orgreater than the threshold value, it is determined that the visual linedirection is changed (Step S14: YES), and the image processing unit 160does not display the display image displayed on the image display unit20.

FIGS. 5A and 5B are explanatory diagrams showing schematic visual fieldsVR which are visually recognized by a user before and after changing thevisual line direction. FIG. 5A shows the visual field VR visuallyrecognized by a user before the visual line direction of a user ischanged. As shown in FIG. 5A, a user visually recognizes a display imageVI. The display image VI includes a pointer PO which moves to beinterlocked with a finger touching the track pad 14. The visual linedirection ED1 of a user is the same direction as the horizontaldirection. FIG. 5B shows a visual field VR visually recognized by a userafter the visual line direction of a user is changed. As shown in FIG.5B, a user glances down, and not in the horizontal direction, and gazesat the control unit 10. In this case, the visual line direction ED1 of auser is changed to a visual line direction ED2. Since an angle α1 formedby the visual line direction ED1 and the visual line direction ED2 isthe changed angle equal to or greater than the preset threshold value,the image processing unit 160 does not display the display image VIafter the visual line direction of a user is changed.

Next, the direction determination unit 166 determines whether or not thepredetermined time (for example, 5 seconds) has elapsed after it isdetermined that the visual line direction of a user is changed (Step S16of FIG. 4). When it is determined that the predetermined time has notelapsed (Step S16: NO), the image processing unit 160 awaits the elapseof the predetermined time. When it is determined that the predeterminedtime has elapsed (Step S16: YES), the image processing unit 160 displaysthe display image VI on the image display unit 20 again (Step S17).Next, when the display image VI is displayed on the image display unit20 (Step S17), or when it is determined that the changed angle of thevisual line direction is smaller than the threshold value and the visualline direction is not changed in the process in Step S14 (Step S14: NO),the control unit 10 monitors the reception of the operation with respectto the operation unit 135 (Step S18). When the reception of theoperation is not detected (Step S18: NO), processes subsequent to StepS12 are repeated. When the reception of the operation such as touchingby a finger of a user on the track pad 14 is detected (Step S18: YES),the control unit 10 performs an operation input process (Step S19).

FIG. 6 is an explanatory diagram showing a flow of the operation inputprocess. In the operation input process, first, the acceleration sensor19 acquires the acceleration acting on the control unit 10 in which theoperation unit 135 is formed (Step S31). Next, the input and outputconversion unit 169 specifies the converted angle based on arelationship between the acquired acceleration and the direction ofgravity (Step S32). Then, the control unit 10 monitors the reception ofthe operation such as the changing of a position of a finger of a userin a state being touched to the track pad 14 of the operation unit 135(hereinafter, also simply referred to as a “pointer operation”) (StepS33). When the pointer operation is received (Step S33: YES), the imageprocessing unit 160 changes the display position of the pointer PO onthe display image VI, based on the received pointer operation (StepS34).

FIG. 7 and FIG. 8 are explanatory diagrams of the changes of the displaypositions of the pointer PO by the pointer operation. FIG. 7 shows alinear line L1 passing through a center O1 of the track pad 14 and acenter O2 of the determination key 11, the direction of gravity DG, andan operation direction DR1 of a forefinger FF of a user moved on thetrack pad 14, in the control unit 10. As shown in FIG. 7, the directionof gravity DG and the operation direction DR1 are the same direction. Inaddition, FIG. 7 shows the visual field VR of a user when the pointeroperation is received. In the embodiment, the converted angle is set as0 degrees as the initial setting, when a reference direction DO from thecenter O1 to the center O2 along the linear line L1 of the track pad 14and the direction of gravity DG are the same direction. Accordingly, theconverted angle is 0 degrees when the operation direction DR1 is thesame direction as the reference direction DO, and thus, the input andoutput conversion unit 169 outputs the movement of the forefinger FFalong the operation direction DR1 as the movement along the operationdirection DR1, in the same manner.

FIG. 8 shows an example in which the input and output conversion unit169 performs correction and output when the reference direction DO andthe direction of gravity DG are not the same direction and the controlunit 10 is inclined with respect to the direction of gravity compared tothe initial setting. As shown in FIG. 8, since the control unit 10 isinclined, the direction of gravity DG and the reference direction DOform an angle β1. The angle β1 is 60 degrees. In this case, the inputand output conversion unit 169 corrects the movement of the forefingerFF in the operation direction DR1 along the reference direction DO, bythe amount of the angle β1 which is the converted angle, by setting aclockwise angle from the direction of gravity DG as a positive angleusing the direction of gravity DG as a reference. That is, even when arelative direction of the operation direction DR1 with respect to thetrack pad 14 is the same as the reference direction DO, the outputsignal converted by the input and output conversion unit 169 isdifferent depending on the direction of gravity DG acquired by theacceleration sensor 19. The OS 150 and the image processing unit 160receive the output signal from the input and output conversion unit 169and change the display position of the pointer PO, as shown in thedisplay image VI of FIG. 8.

When the display position of the pointer PO on the display image VI ischanged (Step S34 of FIG. 6) or the pointer operation is not received(Step S33: NO), the control unit 10 monitors the reception of apredetermined operation with respect to various buttons of the operationunit 135 (hereinafter, also referred to as a “button operation”) (StepS35). When the predetermined button operation is received (Step S35:YES), the control unit 10 performs a predetermined control operationaccording to the button operation (Step S36). When the predeterminedcontrol operation is performed (Step S36) or the button operation is notreceived (Step S35: NO), the control unit 10 monitors the reception ofthe operation for completing the operation input process (Step S37).When the operation for completing the operation input process is notreceived (Step S37: NO), the control unit 10 continuously performs theprocesses subsequent to Step S31. When the operation for completing theoperation input process is received (Step S37: YES), the control unit 10completes the operation input process.

When the operation input process is completed (Step S19 of FIG. 4), thecontrol unit 10 monitors the reception of an operation for completingthe display of the image (Step S20). When the operation for completingthe display of the image is not received (Step S20: NO), the controlunit 10 continuously performs the processes subsequent to Step S12. Whenthe operation for completing the display of the image is received (StepS20: YES), the control unit 10 completes the display image controlprocess.

As described above, in the head mounted display 100 of the embodiment,the acceleration acting on the control unit 10 in which the operationunit 135 is formed, is acquired by the acceleration sensor 19, and theinput and output conversion unit 169 specifies the converted angle basedon the gravitational acceleration acting on the control unit 10, andtransmits the output signal converted from the input received by theoperation unit 135 to the OS 150 based on the converted angle. Inaddition, the 10-axis sensor 66 and the direction determination unit 166specify the visual line direction of a user, and the OS 150 and theimage processing unit 160 control the display image VI displayed on theimage display unit 20 according to the change of the visual linedirection. Accordingly, in the head mounted display 100 of theembodiment, since different control operations are performed inaccordance with the detected results of the plurality of sensors, it ispossible to perform various control operations with respect to the headmounted display 100.

In the head mounted display 100 of the embodiment, the accelerationsensor 19 is formed in the control unit 10 in which the operation unit135 is formed, and acquires the gravitational acceleration acting on thecontrol unit 10. Accordingly, in the head mounted display 100 of theembodiment, since the head mounted display 100 is controlled inaccordance with the input of a user using the direction of gravity as areference, and not the direction of the operation unit 135, a user canperform the input using the direction of gravity as a reference,regardless of the direction of the operation unit 135, and useroperability is improved.

In the head mounted display 100 of the embodiment, when the changedangle of the visual line direction of a user specified by the 10-axissensor 66 embedded in the image display unit 20 and the directiondetermination unit 166 is equal to or greater than the threshold value,the image processing unit 160 does not display the display image VIdisplayed on the image display unit 20. Accordingly, in the head mounteddisplay 100 of the embodiment, since the display image VI is controlledaccording to the visual line direction of a user, sight or the like of auser changes in accordance with the state of a user, and userconvenience is improved.

In the head mounted display 100 of the embodiment, the accelerationsensor 19 acquires the acceleration acting on the control unit 10 inwhich the operation unit 135 is formed, on a regular basis, once every0.5 seconds. Accordingly, in the head mounted display 100 of theembodiment, it is not necessary to constantly acquire the accelerationacting on the control unit 10, and it is possible to suppress the loadapplied to the control unit 10 and to improve user convenience.

In the head mounted display 100 of the embodiment, the display image VIdisplayed on the image display unit 20 is controlled by the OS 150 andthe image processing unit 160, and the input and output conversion unit169 is configured with a device driver. Accordingly, in the head mounteddisplay 100 of the embodiment, since the acceleration sensor 19 and theinput and output conversion unit 169 corresponding to only the operationunit 135 are used, the load of the CPU 140 is decreased, it is notnecessary to change the software such as the OS 150, and it is possibleto reduce a development period of the head mounted display 100.

B. Modification Example

The invention is not limited to the embodiment and can be executed invarious embodiments within a range not departing from the gist thereof,and for example, the following modifications can also be performed.

B1. Modification Example 1

In the embodiment, the 10-axis sensor 66 embedded in the image displayunit 20 detects the state of the image display unit 20, and theacceleration sensor 19 included in the control unit 10 acquires theacceleration acting on the control unit 10, however, variousmodifications can be performed regarding the embodiment of each sensor.For example, the change of the direction of the control unit 10 and theposition of the image display unit 20 is detected by a camera installedon a portion different from the control unit 10 and the image displayunit 20, and the display image VI of the image display unit 20 may becontrolled based on the detected results.

A 10-axis sensor may be embedded in the control unit 10 separately fromthe acceleration sensor 19, instead of the 10-axis sensor 66 embedded inthe image display unit 20. For example, the inputs to the track pad 14may be converted and output by the acceleration sensor 19 and the inputand output conversion unit 169, and the operation such as determinationof an icon included in the display image VI may be performed, accordingto the change of the acceleration detected by the 10-axis sensorembedded in the control unit 10. In the head mounted display 100 of thismodification example, a user can sensorially perform various operations,and user convenience is improved.

In the embodiment described above, the switching is performed to displayor not to display the display image VI of the image display unit 20according to the changed angle detected by the 10-axis sensor 66 of theimage display unit 20, however, various modifications can be performedregarding the content of the head mounted display 100 controlled by thedetected results of the 10-axis sensor 66. For example, when a userlooks down, the display image VI may be displayed at the upper portionin the formed area, and when a user looks up, the display image VI maybe displayed at the lower portion of the area. That is, the displayimage VI may be controlled by the direction of the image display unit 20detected by the 10-axis sensor 66. In addition, as the content to becontrolled by the head mounted display 100, the sound may be output bythe sound processing unit 170 and the earphones 32 and 34, instead ofthe display image VI of the image display unit 20. The control unit 10may perform a control operation for vibrating the image display unit 20.

In the embodiment, the display position of the pointer PO is changed byconverting the pointer operation based on the converted angle, but theregulation for converting the input into the output based on thedirection of the operation unit 135 is not limited thereto, and variousmodifications can be performed. For example, a gyro sensor may beembedded in the control unit 10, instead of the acceleration sensor 19,the angular velocity of the control unit 10 may be detected as thechange of the direction of the operation unit 135, and the display imageVI may be controlled based on the angular velocity. In this case, thecontrol unit 10 itself is horizontally and vertically moved as a stickfor an operation, and the display position of the pointer PO on thedisplay image VI is changed. In this modification example, since a usercan operate the entire control unit 10 including the operation unit 135as one operation unit, an intuitive operation is easily performed, andoperability and user convenience are improved.

In the embodiment, the converted angle is specified based on the angleβ1 between the reference direction DO of the operation unit 135 and thedirection of gravity, however, specifying is not limited to being basedon the direction of the operation unit 135, and various modificationscan be performed. For example, a state where the operation of theoperation unit 135 may be received and a state where the operationthereof may not be received, may be switched, according to the anglebetween an axis orthogonal to the track pad 14 of the operation unit 135and the direction of the gravitational acceleration. In detail, when theangle formed by the direction of an orthogonal axis facing the frontside from the rear side of the track pad 14 and the direction of thegravitational acceleration is equal to or smaller than 90 degrees, thatis, when an operation surface of the track pad 14 faces the direction ofthe gravitational acceleration, it is determined that a user does notperform the operation, and the input and output conversion unit 169 maybe set to a power saving mode in which the operation may not bereceived. In contrast, when the angle formed by the direction of theorthogonal axis and the direction of the gravitational acceleration isgreater than 90 degrees, it is determined that a user is supposed toperform the operation, and the input and output conversion unit 169 maybe set to a standby mode in which the operation may be received. In thismodification example, when the operation surface of the track pad 14faces the direction of the gravitational acceleration, the mode may notbe in the standby mode in which the input may be received, andaccordingly it is possible to suppress power consumption of the headmounted display 100. In this modification example, the angle of 90degrees is described as an example of the angle formed by the orthogonalaxis and the direction of the gravitational acceleration, but the anglethereof is not limited thereto, and various modifications can beperformed. When the operation unit 135 is operated, the angle may bearbitrarily set. The state where the angle formed by the direction ofthe orthogonal axis and the direction of the gravitational accelerationin the modification example is equal to or smaller than 90 degrees,corresponds to a first state in the claims, and the state where theangle thereof is greater than 90 degrees corresponds to a second statein the claims.

The regulation for converting the input into the output based on thedirection of the operation unit 135 may be arbitrarily changed byreceiving the predetermined operation. For example, the conversion basedon the converted angle specified according to the direction of theoperation unit 135 and the setting of the state based on the angleformed by the direction of the orthogonal axis and the direction of thegravitational acceleration, may be changed based on the operationreceived by the operation unit 135. In addition, infrared light emittedby an infrared light emitting unit formed in the image display unit 20is received by an infrared light receiving unit formed near the trackpad 14, and accordingly, even when the direction of the orthogonal axisand the direction of the gravitational acceleration are the same, theoperation unit 135 may be set to a state where the operation may bereceived. In this case, it is even possible to operate the head mounteddisplay 100 in a state where a user lies around, with operability anduser convenience being improved. The same state may be set by imagingthe control unit 10 by a camera formed in the image display unit 20,instead of the infrared light emitting unit and the infrared lightreceiving unit of this modification example. Various modifications canbe performed regarding the setting for changing the regulation forconverting the input into the output according to a positionalrelationship between the direction of the image display unit 20 and thedirection of the operation unit 135. The state where the infrared lightemitted by the infrared light emitting unit of this modification exampleis received by the infrared light receiving unit, corresponds to a firststate in the claims, and the state where the infrared light is notreceived by the infrared light receiving unit, corresponds to a secondstate in the claims.

B2. Modification Example 2

In the embodiment, the input and output conversion unit 169 corrects theoutput by the amount of the angle β1 of the converted angle formed bythe reference direction DO and the direction of gravity DG, but variousmodifications can be performed regarding the converted angle to becorrected. For example, the input and output conversion unit 169 maycorrect the output based on the four converted angles, according to theangle β1 formed by the reference direction DO and the direction ofgravity DG. When the angle β1 is from 0 degrees to 45 degrees or from315 degrees to less than 360 degrees, the input and output conversionunit 169 sets the converted angle to 0 by assuming the referencedirection DO and the direction of gravity DG to be the same directions.In the same manner as described above, the input and output conversionunit 169 may set the converted angle to 90 degrees when the angle β1 isgreater than 45 degrees and equal to or smaller than 135 degrees, mayset the converted angle to 180 degrees when the angle β1 is greater than135 degrees and smaller than 225 degrees, and may set the convertedangle to 270 degrees when the angle β1 is equal to or greater than 225degrees and smaller than 315 degrees. In this modification example,since there are only four converted angles, it is easy to correct theinput received by the track pad 14 to the output, it is possible todecrease the load of the system, and it is possible to reduce thefrequency of acquiring the acceleration acting on the control unit 10 bythe acceleration sensor 19. With the input received by the track pad 14,the input received by the direction key 16 may be corrected to theoutput based on the converted angle. In this modification example, theinput and output conversion unit 169 divides the angle β1 into fourranges, and corrects the input received by the track pad 14 to theoutput according to the angle β1.

In the embodiment, when the reference direction DO and the direction ofgravity DG are different from each other, the output is corrected by theamount of the angle β1, but the correction of the output may not beperformed, even when the reference direction DO and the direction ofgravity DG are different from each other. For example, the menu key 17or the like of the operation unit 135 is pressed down for apredetermined time or longer (hereinafter, also referred to as a “longpress”), and accordingly, the converted angle at a time point when thelong press is started may be maintained. In this modification example,for example, even when a state of a user is changed from the sittingstate to a laying-down state, and the operation unit 135 is operated inthe laying-down state, the relative direction of receiving the operationof the operation unit 135 with respect to a user is not changed, andaccordingly user convenience is improved.

B3. Modification Example 3

In the embodiment, the input and output conversion unit 169 is set asthe device driver which transmits the output signal to the OS 150, butvarious modifications can be performed regarding the configuration ofthe input and output conversion unit 169. For example, the input andoutput conversion unit 169 may be configured between the device driverand the OS 150 in a layered structure of the system, or may beconfigured in middleware which is between the hardware and the software(for example, hardware abstraction layer (HAL)). The operating system inthe claim is software which provides basic functions commonly used formultiple application software, such as the input received by theoperation unit 135, the input and output functions for displaying thedisplay image VI, or management of a disk or a memory, and manages theentire computer system. In addition, the device driver in the claim issoftware which controls or operates the device mounted in the computeror a device connected to the outside. Further, the middleware in theclaim is software which is operated on the OS 150 and provides advancedspecific functions other than the OS 150 with respect to the applicationsoftware.

In the embodiment, the head mounted display 100 is described as anexample of the information processing apparatus including the operationunit, however, it does not necessarily have to be the head mounteddisplay, and various modifications can be performed regarding theembodiment of the information processing apparatus. For example, theinformation processing apparatus may be a device including a devicewhich displays an image on a display disposed on a monitor, instead ofthe image display unit 20.

B4. Modification Example 4

In the embodiment, the operation unit 135 is formed in the operationunit 10, however, various modifications can be performed regarding theembodiment of the operation unit 135. For example, a user interfacewhich is the operation unit 135 may be used separately from the controlunit 10. In this case, since the operation unit 135 is providedseparately from the control unit 10 in which the power 130 or the likeis formed, it is possible to perform miniaturization and useroperability is improved.

For example, the image light generation unit may have a configurationincluding an organic electro-luminescence (EL) display and an organic ELcontrol unit. For example, as the image light generation unit, a liquidcrystal on silicon (LCOS; LCoS is trademark) or a digital micro mirrordevice can be used, instead of the LCD. For example, the invention canalso be applied to a laser retina projection type head mounted display100.

For example, the head mounted display 100 may be a head mounted displayhaving the embodiment in which the optical image display unit onlycovers a part of the eyes of a user, that is, the embodiment in whichthe optical image display unit does not entirely cover the eyes of auser. In addition, the head mounted display 100 may be a so-calledsingle eye type head mounted display. The head mounted display 100 isset as a both eye type optical transmission type device, but theinvention can also be applied to a head mounted display of another typesuch as a video transmission type device, for example.

In addition, as the earphone, an ear wearable type or a head band typedevice may be used, or the earphone may be omitted. For example, a headmounted display loaded on a vehicle such as a car or an airplane may beconfigured. Ahead mounted display embedded in a body protecting devicesuch as a helmet may be configured.

B5. Modification Example 5

A configuration of the head mounted display 100 of the embodiment is oneexample, and various modifications can be performed thereon. Forexample, one of the direction key 16 and the track pad 14 provided inthe control unit 10 may be omitted, or the other operation interfacesuch as a stick for an operation may be provided in addition to thedirection key 16 and the track pad 14 or instead of the direction key 16and the track pad 14. The control unit 10 may be configured to beconnected to an input device such as a keyboard or a mouse so as toreceive the input from the keyboard or the mouse.

As the image display unit, instead of the image display unit 20 mountedas the glasses, an image display unit of another type such as an imagedisplay unit mounted as a hat, for example, may be used. The earphones32 and 34 or an external scenery imaging camera 61 can be suitablyomitted. In the embodiment, the LCD and the light source are used as theconfiguration for generating the image light, but instead thereof,another display element such as an organic EL display may be used. Inthe embodiment, the 10-axis sensor 66 is used as the sensor fordetecting the movement of the head of a user, but instead thereof, asensor configured with one or two of an acceleration sensor, an angularvelocity sensor, a geomagnetism sensor, and an atmosphere sensor may beused.

FIGS. 9A and 9B are explanatory diagrams showing appearanceconfigurations of the head mounted display of the modification examples.In a case of an example of FIG. 9A, the differing points from the headmounted display 100 shown in FIG. 1 are that an image display unit 20 aincludes a right optical image display unit 26 a instead of the rightoptical image display unit 26 and includes a left optical image displayunit 28 a instead of the left optical image display unit 28. The rightoptical image display unit 26 a is formed to be smaller than the opticalmember of the embodiment described above, and is disposed at anobliquely upper portion of the right eye of a user when mounting a headmounted display 100 a. In the same manner as described above, the leftoptical image display unit 28 a is formed to be smaller than the opticalmember of the embodiment described above, and is disposed at anobliquely upper portion of the left eye of a user when mounting the headmounted display 100 a. In a case of an example of FIG. 9B, the differingpoints from the head mounted display 100 shown in FIG. 1 are that animage display unit 20 b includes a right optical image display unit 26 binstead of the right optical image display unit 26 and includes a leftoptical image display unit 28 b instead of the left optical imagedisplay unit 28. The right optical image display unit 26 b is formed tobe smaller than the optical member of the embodiment described above,and is disposed at an obliquely lower portion of the right eye of a userwhen mounting a head mounted display. The left optical image displayunit 28 b is formed to be smaller than the optical member of theembodiment described above, and is disposed at an obliquely lowerportion of the left eye of a user when mounting the head mounteddisplay. As described above, the optical image display units are notlimited as long as they are disposed near the eyes of a user. The sizeof the optical members configuring the optical image display units isalso arbitrarily set, and it is possible to realize the head mounteddisplay 100 having the embodiment in which the optical image displayunit covers only a part of the eyes of a user, that is, the embodimentin which the optical image display unit does not entirely cover the eyesof a user.

In the embodiment, the head mounted display 100 may guide the imagelight showing the same images to the right and left eyes of a user toallow a user to visually recognize a two-dimensional image, or may guidethe image light showing the different images to the right and left eyesof a user so as to allow a user to visually recognize athree-dimensional image.

In the embodiment, a part of the configuration realized with thehardware may be replaced with the software, and conversely, a part ofthe configuration realized with the software may be replaced with thehardware. For example, in the embodiment, the image processing unit 160or the sound processing unit 170 is realized by reading out andexecuting the computer programs with the CPU 140, however, thefunctional units may be realized by a hardware circuit.

When some or all of the functions of the invention are realized with thesoftware, the software (computer programs) can be provided in a state ofbeing stored in a computer-readable recording medium. In the invention,the “computer-readable recording medium” is not limited to a mobilerecording medium such as a flexible disk or a CD-ROM, but also includesvarious internal recording devices in the computer such as a RAM or aROM, or external recording devices fixed to the computer such as a harddisk.

In the embodiment, as shown in FIG. 1 and FIG. 2, the control unit 10and the image display unit 20 are formed with separate configurations,however, the configurations of the control unit 10 and the image displayunit 20 are not limited, and various modifications can be performed. Forexample, the entirety or a part of the configuration formed in thecontrol unit 10 may be formed in the image display unit 20. The power130 of the embodiment may be formed alone to be replaced, or may beformed in the image display unit 20 with the same configuration formedin the control unit 10. For example, the CPU 140 shown in FIG. 2 may beformed in both of the control unit 10 and the image display unit 20, orthe functions performed by the CPU 140 formed in the control unit 10 anda CPU formed in the image display unit 20 may be configured separated.

The control unit 10 may be embedded in the PC and the image display unit20 may be used instead of the monitor of the PC, or the control unit 10and the image display unit 20 may be integrally provided to be attachedto the clothing of a user as a wearable computer.

The invention is not limited to the embodiments and modificationexamples, and can be realized with various configurations within a rangenot departing from the scope thereof. For example, the technicalfeatures in the embodiments and the modification examples correspondingto technical features in each embodiment disclosed in the summary of theinvention can be suitably replaced or combined, in order to solve someor all of the problems described above or in order to realize some orall of the advantages described above. The technical features can besuitably removed, as long as the technical features are not described ascompulsory in the specification.

What is claimed is:
 1. An information processing apparatus comprising:an operation receiver which receives an operation; and a processorprogrammed to detect a direction of the operation receiver based on asignal from one or more sensors, convert an input of the operationreceived by the operation receiver into an output based on a firstregulation, when the direction of the operation receiver is in a firststate, and convert the input into an output based on a second regulationwhich is different from the first regulation, when the direction of theoperation receiver is in a second state which is different from thefirst state, detect an apparatus state which is at least one of aposition and a direction of the information processing apparatus basedon a signal from the one or more sensors, and perform a control processof the information processing apparatus, based on the apparatus state,wherein when an angle formed by a direction of an orthogonal axis facinga front side from a rear side of the operation receiver and a directionof a gravitational acceleration is equal to or smaller than apredetermined angle, the processor is set to a first mode in which theoperation is not received, and when the angle is greater than thepredetermined angle, the processor is set to a second mode in which theoperation is received.
 2. The information processing apparatus accordingto claim 1, wherein the processor is disposed in the operation receiverand further programmed to detect the direction of the operation receiverusing the acceleration of the operation receiver.
 3. The informationprocessing apparatus according to claim 2, wherein the first state andthe second state are determined based on a relationship between thedirection of the operation receiver and the direction of gravity.
 4. Theinformation processing apparatus according to claim 3, wherein the firststate is a state where an operation angle formed by the direction of theoperation receiver and the direction of gravity is equal to or smallerthan 45 degrees, and the second state is a state where the operationangle is greater than 45 degrees.
 5. The information processingapparatus according to claim 4, wherein the second state includes athird state where the operation angle is greater than 45 degrees andequal to or smaller than 135 degrees, and a fourth state where theoperation angle is greater than 135 degrees, and the processor isfurther programmed to convert the input of the operation received by theoperation receiver into an output based on a third regulation differentfrom the first regulation, in a case of the third state, and convert theinput into an output based on a fourth regulation different from thefirst regulation and the third regulation, in a case of the fourthstate.
 6. The information processing apparatus according to claim 1,further comprising: an image display which forms image light based onimage data and allows a user to visually recognize the image light as avirtual image, in a state of being mounted on the head of a user,wherein the apparatus state is at least one of a position and adirection of the operation receiver or the image display, and theprocessor is further programmed to control the image light formed by theimage display, as the control process.
 7. The information processingapparatus according to claim 6, wherein the processor is disposed in theimage display, and is further programmed to detect at least one of theposition and the direction of the image display, and set a position ofthe image light formed in the image display, as the control process. 8.The information processing apparatus according to claim 1, wherein theprocessor is further programmed to detect the direction of the operationreceiver with a frequency of less than once every 0.5 seconds.
 9. Theinformation processing apparatus according to claim 1, wherein theprocessor includes an operating system; and a device driver ormiddleware.
 10. The information processing apparatus according to claim1, wherein when the direction of the orthogonal axis and the directionof the gravitational acceleration are the same and an image display anda controller, in which the operation receiver is disposed, face eachother, the processor is set to a standby mode.
 11. The informationprocessing apparatus according to claim 10, wherein an infrared lightsender and an infrared light receiver or a camera formed in the imagedisplay is configured to determine when the image display and thecontroller face each other.
 12. The information processing apparatusaccording to claim 1, wherein when a user input of a predeterminedoperation is received, a converted angle at a time point when the userinput of the predetermined operation is started is maintained theconverted angle being an angle specified based on the direction of thegravitational acceleration and a reference direction of the operationreceiver.
 13. The information processing apparatus according to claim 1,further comprising: an image display, wherein when a user changes avisual line direction, an image is not displayed by the image displayand when a determination is made that a predetermined time has elapsedafter a determination is made that the visual line direction is changed,the image is displayed by the image display.
 14. A control method of aninformation processing apparatus including an operation receiver whichreceives an operation, and a processor programmed to detect a directionof the operation receiver based on a signal from one or more sensors,and detect an apparatus state which is at least one of a position and adirection of the information processing apparatus based on a signal fromthe one or more sensors, the method comprising: converting an input ofthe operation received by the operation receiver into an output based ona first regulation, when the direction of the operation receiver is in afirst state, and converting the input into an output based on a secondregulation which is different from the first regulation, when thedirection of the operation receiver is in a second state which isdifferent from the first state; and performing a control process of theinformation processing apparatus, based on the apparatus state, whereinwhen an angle formed by a direction of an orthogonal axis facing a frontside from a rear side of the operation receiver and a direction of agravitational acceleration is equal to or smaller than a predeterminedangle, the processor is set to a first mode in which the operation isnot received, and when the angle is greater than the predeterminedangle, the processor is set to a second mode in which the operation isreceived.